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Bidirectional associative memory (BAM) is a kind of an artificial neural network used to memorize 
and retrieve heterogeneous pattern pairs. Many efforts have been made to improve BAM from the 
the viewpoint of computer application, and few theoretical studies have been done. We investigated 
the theoretical characteristics of BAM using a framework of statistical-mechanical analysis. To 
investigate the equilibrium state of BAM, we applied self-consistent signal to noise analysis (SCSNA) 
and obtained a macroscopic parameter equations and relative capacity. Moreover, to investigate 
not only the equilibrium state but also the retrieval process of reaching the equilibrium state, we 
applied statistical neurodynamics to the update rule of BAM and obtained evolution equations for 
the macroscopic parameters. These evolution equations are consistent with the results of SCSNA 
in the equilibrium state. 



I. INTRODUCTION 

Bi-directional associative memory (BAM) [2 is a kind of an associative memory model which is an 
artificial neural network. The principle function of associative memory is to memorize multiple patterns and 
to retrieve the correct one when a pattern key is given. 

Autocorrelation associative memory (AAM), sometimes called the Hopfield model 0, is also a kind 
of associative memory. AAM tries to retrieve a stored pattern when a degraded pattern is given as an 
association key; this type of retrieving is called homogeneous association. In contrast, BAM stores multiple 
pattern pairs and tries to retrieve a complete stored pattern pair when a degraded piece of the pair is given 
as an association key. Thus, BAM is called a heterogeneous pattern association model. 

In the field of neural networks, many efforts have been made to improve BAM from the viewpoint of 
computer application HH'B^'e^lllldll^, and few theoretical analyses have been reported [l^ 
[l3l| . The theoretical analysis of BAM has evolved with a focus on storage capacity, which means how many 
patterns can be stored in a network consisted of N neural units. Yanai et al. s ugg ested that BAM can be 
regarded as a variation of AAM in which connections are systematically reduced [ij . They also showed that 
the relative storage capacity, in which a finite amount of retrieval error is allowed, of BAM to be around 
0.22iV. Haines & Hecht-Nielsen analyzed BAM along the same way, and reported its absolute capacity, in 
which no retrieval error is allowed, to be 0(iV/logiV) [l^. Tanaka et al. analyzed BAM using a replica 
method (see ^3); which is a statistical- mechanical analysis method, and showed its relative capacity to be 
0.19987V |fl3|. These analyses mainly focused on the equilibrium state of BAM, and the transient process of 
retrieving, which means how to reach the equilibrium state, was not so conducted. However, analysis of the 
retrieval process is as important as that of the equilibrium state. 

In this paper, we have analyzed the equilibrium state of BAM using the self-consistent signal-to-noise 
analysis (SCSNA) [l5| . We found that the relative capacity was 0.19987V, which agrees with the result of 
Tanaka et al. We also investigated the retrieval process of BAM; we derived macroscopic dynamical equations 
using the statistical neurodynamics, which was theoretically derived in the same manner as the SCSNA [l| 
[r^ , and compared the results of between the statistical neurodynamics with those of computer simulation. 
Applying the statistical neurodynamics to BAM, we obtained the evolution equations for the macroscopic 
parameters. In the limit of these evolution equations, that is, macroscopic parameters of BAM reached 
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the equilibrium state, we found these values were consistent with the results of SCSNA. We also compareS 
the results of applying the statistical neurodynamics with those of the computer simulation and obtained 
quantitative support for our analysis. 

We describe the BAM formulation in Section (nj and we apply the SCSNA and show the results of 
equilibrium state analysis in Section IIIII In Section IIVI we derived the evolution equations of macroscopic 
parameters using the statistical neurodynamics and compared the results with those of computer simulation 
m 

Section El 



II. FORMULATION 

As shown in Fig. Q BAM is a two-layered neural network model [H. The first layer consists of cN neural 
units (c ~ 0(1)), and the state of the layer is denoted as x with the components denoted as Xi {1 < i < cN). 
The state of the second layer, which has cN units, is denoted as 5, and the jth unit state is described as 
(1 5: i !i cN). Each layer is connected by interlayer connection J with the components described as 
Jij (1 < i < cN, 1 < J < cN). Jij represents the connection weight between the first layer unit Xi and the 
second layer unit Xj. 

We prepare p binary pattern pairs denoted as {^'^,1^^} (m = 1, • ■ ■ where the superscript /j, denotes 
the pattern pair index. Pattern vector corresponds to the first layer, and corresponds to the second 
layer. Thus ^'^ and have cN and cN components, respectively, and each component, which is described 
as and (1 < i < cN, I < j < cN), respectively, is generated from uniform i.i.d.: 

Prob[C^ = ±1] = i (1) 
ProbK;±l] = i. (2) 

Assuming the number of stored pattern pairs to be p ~ 0{N), we define a quantity a (= -^), and use it for 
the loading rate. 

To determine the interlayer connection weight Jij , which connects Xi and Xj , we use a correlation-based 
learning rule: 

aN 

■^^^=j^j:m- (3) 

All the pattern pair correlations between and are embedded in connection weight J. In this notation, 
the connections are not symmetrical, that is Jij ^ Jji. 

In the retrieving, we use a synchronous update rule for each layer; that is, all units in each layer are 
updated synchronously, and these layers are updated alternately. The rules for updating the ith unit in the 
first layer and the jih unit in the second layer are 

cN 

„2t 



= F(^J.,5f-i), (4) 



x^ 

cN 



i=F(^J,,-a:f), (5) 



where t means the one step time and F(-) means the output function. In these formulations, the retrieval 
process is carried out as follows. In a initial state, i = 0, Association key x^{— {x"}) is given to the first 
layer. Then, all of the second layer units, (1 < j < cN), are updated using eq. and the state of the 

second layer is described as x} . Next, t = 1, all the units in the first layer, xj [1 < i < cN), are updated 
using eq.Q), and the state is described as x^. After that the second layer is updated by eq.©. For each 
t — 2,3, ■ ■ ■ , the alternate updating of each layer are carried out in the same way, and each layer state is 
denoted as x^^ and 5^*"*"^. 



To apply S/N analysis, we introduce overlaps, which means similarities between patterns. The overlap^ 
between first layer state x^* and the fith pattern, and between second-layer state 5^*"^ and are 
described as follows, respectively: 



^ cN 

i=l 
cN 



Following the S/N analysis, we decomposed the inner term of F{-) in eqs.Q) and Q into signal and noise 

components. Assuming the first pattern pair, {^^, ^ }, is retrieved, the terms including overlaps mi and rhi, 
are signal components, i.e. mi, rhi ^ 0(1). Using these overlaps, eqs.(@J and © can be described as 

xr=F{&n^''^l+z^'-'), (8) 
=F(cmff;i+5f), (9) 



a; 



where zf*~^, and Zj* are called as crosstalk noises, which prevents the target pair {^^,^ } to be retrieved. 
These crosstalk noises are denoted 



r"^ = ^EE^r^^r' (10) 

aN cN 



J N 



III. EQUILIBRIUM STATE ANALYSIS BY SCSNA 

To derive equilibrium state macroscopic parameters, we use SCSNA TBI, which is an extension of a naive 
signal-to- noise (S/N) analysis. Since the SCSNA treats the equilibrium states of an associative memory 
model, we omit the index t in the update rules. Hence we can rewrite eqs. ijHl and as 

X, = F{dmi^l + z,), (12) 
Xj = F(cmi^j -I- Zj), (13) 

respectively. 

In the SCSNA, the crosstalk noise term is decomposed into a systematic bias term and a Gaussian noise 
term with mean|0|. The detailed formulas of SCSNA are described in appendix. We derive self-consistent 



equations called order parameter equations. The following are the order parameter equations of BAM. ^ 

y = F(cmC + - —^Y + ^z), (14) 

1 — ccUU 

Y = F(cmi+ — . Y + V^z), (15) 
1 — ccUU 

m = j Dzi^Y}^, (16) 

DziiY)^, (17) 



q = jDz{Y^)^, (18) 
q=fDz{Y^)^, (19) 



U=^[dzz{Y)^, (20) 
VQr J 

U=-^J Dzz{Y)^, (21) 



r=- ^—^{<l + ccU\), (22) 

(1 — ccuuy 

f=- %-^^{q + cW^q). (23) 

(1 — caJUy 

These equations are described in the manner of Shiino and Fukai p^ . Y and Y represents the effective 
outputs for Xi and a;-,-, respectively. The stochastic variables ^ and ^, obeying eq.(^ and (0), corresponds to 
a retrieving pattern components and ^j, and order parameters m and m corresponds to overlaps mi and 
TOi. Note that the operators (•)^ and (•)| mean the expectations for stochastic variables ^ or ^, respectively. 
Each arguments of the function _F'(-) consists of three parts. The first terms, cm^ and cm^, come from the 
signal components, the second terms, — ^^tttv^ and — °"]^,,-, Y, mean the systematic bias of the crosstalk 

' ' l~ccUU l-ccUU ' 

noises (z^, Zj) in eqs. (|10|l and and each third term is assigned to be a Gaussian distribution with 

mean and ar or af variance. We solved the order parameter equations from H14|l to H23|l numerically and 
compared the results with those of simulations. Fig. 13 shows the equilibrium overlap m against the capacity 
parameter a. An overlap of 1 means that the BAM retrieves a stored pattern pair successfully. We obtained 
a relative capacity, ac of 0.1998 in which the nontrivial solution m ^ and to 7^ is disappeared. This 
agrees with the results of Tanaka et al. {uc = 0.1998), obtained with the replica method [1^. In fig|21 we 
show the simulation results as error-bars, which mean medians and quartile deviations for ten trials. The 
SCSNA results quantitatively explained the simulation results very well. 



IV. RETRIEVAL PROCESS OF BAM 



As we have seen, the SCSNA described the equilibrium state of BAM quantitatively. In this section, we 
consider a retrieval process of BAM, which means the transient process reaching the equilibrium state. The 
statistical neurodynamics, which is a theory for the retrieval for associative memory model, is based on S/N 
analysis. Amari and Maginu proposed a statistical neurodynamical theory on the S/N analysis It was 
known that the storage capacity obtained by Amari & Maginu theory does not coincide with the results of 
the replica theoryj^, and the size of the basin of attraction derived from Amari & Maginu theory is larger 
than the results of the computer simulation. Okada extended the Amari & Maginu theory to improve to 
resolve these difficulties (17i|. and obtained a macroscopic equation which has hierarchical structure. In the 
macroscopic equation, the first-order approximation corresponds to the Amari & Maginu theory, and the 
higher order approximation coincide with the replica theory. 

For applying the statistical neurodynamics to BAM, we evaluate the crosstalk noises feas. (|10f) and (|ll|l ) 

in eqs. l|SJl and ©. Assuming the first pattern pair, }, is retrieved, we can regard the overlaps of 



other pattern pairs, {m^*, m^/"*"^} where /i > 2, as small. Thus, we expand the state x^* and 



2t z.2t+l. 



aN 



aN 
aN 



(24) 
(25) 



aN 



for fi>2, where ^ ^(^^^2*1^ ^2tM ^ p(^~^^2t-i^uy Substituting eqs. ^ and ^ 

into eas. l(Tn|) and ((TT)l . we obtain. 



5f = acC/2*x7-^^(''^+Zf, 



where 



aA^ cA'^ ~ " aN cN 

1 \^ CMgM;^2t + l,(;j) c£/2f+l 2t,(M) , ~fr rr y2t-l 

Ai=2 j=l ^1=2 fe^^i 



aAf cN rr aN cN 

CU' 



^=2 i=l 



TV E E ^j*^; -^i 

M=2 l^j 



where 



cN aN 



U2t+1 



U2: 



cN aN 

-^E^'(^E-r^^r). 



(26) 
(27) 



(28) 
(29) 

(30) 
(31) 



Since xf'^^'' and are almost independent with and ^j*, respectively, each Zf'^^ and Z|* can be 

regarded as independent identical Gaussian distributions, that is Z^^*^^ ~ -^(0, ar2t+i) and Z^* ^ ^(0, a?^2t)- 
Each noise variance, E^Zf*^^)"^] ~ ar2t+i and £^[(Z|*)^] — ar2t, can be described as 



ar2t+i = acq2t+i + acc ?72t+i92i + a{ccU2t+iU2t) ?'2t-i 



2c£C/2t+lC/2t£^ 



p=2j=l 



af2t = acg2i + acc f72t92t-i + Q!(ccC/2tC^2t-i) ^"24-2 

qTV cN 



2cW2tU2t^iE 



p=2 i=l 



where 



1 /~2t+l,(^ 



(32) 
(33) 

(34) 
(35) 



The last terms in eas. (|32ll and H33|l are determined by correlations between the current state il*"*"^ and th§ 
previous state noise variable Zf*^^, and between a;f* and respectively. Assuming that the {n + 1) 

previous state noise variables and ^2(t-n)-2 j^^^g correlation with the current state x'^*'^^ and 

xf*, respectively, we can expand r2t+i and r2t as recurrence formulas: 

r2t+i = cq2t+i + c{cU2t+if'q2t + {ccU2t+lU2tfr2t-l 
n t 

+ 2c^(cc)''g2t+i,2(t-r,)+i W U2T+1U2T 

7] — ! T — t — 7]-\-l 

n-1 t 
+ 2c(c;72t+l)';^(c5)"g2t,2(t-r,) n U2rU2r-l, (36) 

7] — ! T=t — 77+1 

ht = Cq2t + c(cC/|t)^(72t-l + (ccC/2tC/2t_l)^f2t-2, 

+ 2c^(cc)''g2t,2(t-r,) n ^2Tt^2T^l 
r/— 1 r— t — 77+1 

n-1 t 

+ 2c{cU2tf ^(5c)''g2t»l,2(t~„)-in C^2r-lC/2r-2, (37) 
rj—1 T—t — rj+1 

where q'2t+i,2(t-n)+i nieans a cross-correlation between the current state i^*'''^ and the n-step previous 

state x^^*""^'*'^, and q2t.2(t-n) means a cross-correlation between xf* and xf'^~^\ These variables can be 
also described with the macroscopic parameters across the n-step previous state. The complete formula is 
described in the appendix. 

We obtain the evolution equations for macroscopic parameters as follows: 

r^* = F(5m2t-iC + Va^^z), (38) 



f 2t+i ^ F(^cm2i + Vahtz), (39) 
m^* = / Dz{^Y'')^, (40) 



m2*+i = / (41) 



to = J Dz{iY'r)i, (42) 
g2t+i = / Dz{{Y''+'f),, (43) 



t/2t - / i5zz(F2t) (44) 

(72t+i = / i?zz(P*+i) - (45) 



In these order parameter equations, F^* and F2t+i correspond to the xf* and ij*"''"'^, respectively. The 



overlaps for the first pattern pair, mf* and m^*"''^, which mean retrieval degree, correspond to the m^* and 
^2t+i^ respectively. 

Yanai et al. jlj applied the one-step analysis, which corresponds to Amari & Maginu theory. In their 
analysis, the macroscopic order parameter equations for K^^*, Y^*^^ , mf*, m^*^^, q2t, q2t+i, U2t,U2t+i, which 
are described in eqs. from (|38|l to (|45|l . are identical to those of our analysis. The differences are in evaluating 
of noise variances, that is, r2t+i and r2t- They ignored the noise correlation, and derived these values as 

7-2i + l = cq2t+i + c{cU2t + i)^q2t (46) 
■r2t = cq2t + c(cU2tfq2t-i- (47) 



In their result, the critical capacity ac is equal to 0.27, which is not equal to our SCSNA analysis and thi 
replica analysis [ac = 0.1998 for both analyses). This overestimation comes from the lack of noise correlation 
evaluation. 

In our analysis, we consider the effect of crosstalk noise correlation across 71-step previous state, and 
obtained the eas. l|32|) and H'3'3\i which includes Yanais' analysis feas. (|46|l and (|47|) '). In the next section, we 
show that the analysis accuracy improves as n in increased (rt = 2, 3, • • • ). Hereafter, we call the statistical 
neurodynamics considering across the n-step previous state as the "n-step" analysis in the following. "Full- 
step" analysis means using all the macroscopic parameters from the initial state {t = 0) to the current 
state. 



V. RESULT 

In this section, we compare the results of the statistical neurodynamics with those of computer simulation. 
Fig. 121 shows the time evolution of the overlap m^*, which means how well the pattern is retrieved in 
the first layer at 2t. Each abscissa axis represents the time step t and each ordinate axis describes overlap 
mf*. Convergence of the overlap mf* to 1.0 means success in retrieving In the graphs, we show several 
evolution curves in which the initial overlap (m^) starts with a different state. 

Fig. Ela) shows the simulation results. We set the number of neurons N to 10,000, c — c — 1, and the 
number of stored pattern pairs was indexed as a = 0.15. The retrieval was successful when we set the initial 
overlap larger than 0.4, and it failed when we set it to 0.3 or less. Fig. |SIb) shows the results of the one-step 
analysis, and figsOfc) to (e) shows the result of the 2-step, 3-step, and full step analysis, respectively. In 
each analysis result, the overlap converged to when retrieving failed because of assuming infinite neuron 
units {N — > 00) exist. In the simulation results, figEla), the system settled into a spurious memory state 
when retrieving failed because the number of units was finite {N = 10,000). Therefore, the curves starting 
at 0.1 to 0.3 can be regarded as retrieval failures. 

As shown in fig. |3Jb), the one-step analysis says that retrieving is successful when the initial overlap is 
0.3, which does not agree with the simulation results. Fig. ^c) shows the results for n — 2, i.e. the 2-step 
analysis. The 2-step analysis says that the retrieval is a failure when the initial overlap is 0.3, which agrees 
with the simulation results. Fig. OJd) and (e) show the 3-step results and the full-step analysis results, 
respectively. Each figure shows similar characteristics, and the results agree with those of the simulation 
results shown as fig. Ol^a). Since the 3-step analysis results are very similar to the full-step analysis results 
the 3-step analysis is enough for approximating the full-step analysis. In other words, the previous 3-step 
correlations are effective for BAM retrieval. 

In the statistical neurodynamics analysis, the equilibrium state is described as the limit of the transient 
process, and the order parameters should be consistent to the result of the SCSNA. Fig. 2|shows the memory 
capacity and the basin of attraction , which means the degrading limit of the retrievable pattern in the initial 
state measured by the overlap mj. For example, in fig. Old), the retrieval is successful when starting at 
m° — 0.4, while it is a failure when starting at = 0.3. There is thus a basin when is between 0.3 and 
0.4 for a = 0.15. The dashed curves in fig0]are derived from the statistical neurodynamicses. In these curves, 
the upper part shows equilibrium overlap in successful retrieval and the lower part shows the basin of 
attraction m^. When we set the initial overlap ml to be in the area surrounded by these curves, the retrieval 
will be success. Therefore, the area surrounded by these curves represents the successful retrieval area. It is 
clear that the one-step analysis overestimates both the relative capacity and the basin of attraction^^- The 
theoretical estimation accuracy improves and comes close to that of the SCSNA analysis asymptotically as 
the analysis accuracy is improved (2-step, 3-step, ■ ■ • .) We also show the basin derived from the simulation 
results using error-bars in fig01 The results of simulation agree with those of the statistical neurodynamics 
quantitatively. 

VI. CONCLUSION 

We derived the macroscopic parameters of BAM in the equilibrium state by using the SCSNA and obtained 
the critical capacity ac as 0.1998. The results agreed with the previous results and the simulation results. 

We also analyzed the transient process of BAM using the statistical neurodynamics and obtained the evo- 
lution equations for the macroscopic parameters. Comparison of the numerical solutions with the simulation 



results, we showed that the analysis results can explain the simulation results with sufficient accuracy fo§ 
the transient process. Therefore, to explain the transient process of BAM quantitatively, it is sufficient to 
consider the 3-step statistical neurodynamics, which means that the crosstalk noise has effective correlation 
across the 3-step previous state. 
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APPENDIX A: DETAIL SCSNA DESCRIPTION 



In Sec lllll we introduced the overlaps, 



cJV 



i — l 
cN 



for each layer state. In the equiUbrium state, we assumed that the first pattern pair, (^^, f ), is retrieved, so tha? 
overlaps for other pattern pairs are small, i.e. m^,rhn ~ 0{^^) where n>2. Thus we denotes the nip. 



^^ = 7^Il^"n^Jl^"^^) (A3) 



i— 1 ^t — 1 

cN olN . cN 



E ^^^^(^E ^'^^) + ^ E crh,F'{cJ2^rrh.) (A4) 

i—l i— 1 i^^f-i- 

+ cm^ i7, (A5) 



where 

cJV qJV 

'^=^E^'(5E«>^) (A6) 

■i—l 

cN aN cN 

We denote the x^^^ as the value drawn the effect of /ith pattern pair from Xi, i.e. Xi — x\^^ ~ 0(-^). m^j is also 
denoted 

~ + cmpC/, (A8) 

where 

^ cA^ aN 

z — 1 J^^f^ 
cN 

^M = ^E4"4'^^- (AlO) 
Solving egs. lASt and 1A8II for m'' and m'', we obtain 

;{M,, + cUMp), (All) 
;(M„ + ci/Mp). (AI2) 



I - ccUU 
1 



I - cc(/f/ 

aN aN 

Since the noise terms in eas. l|12|l and H13^ can be described as Zi = ^rn^i^, and 5j = Ci""^" j substituted 
egs. lAlU and IIA12II to these noises and obtained 

-xf'+Z,, (AI3) 



acU jp) 

1 - ccUU ' 
acU _(^) 

1-ccUU ' 

where 

aN I cN 



(AI4) 



cJV 



We assumed Zi and as independent identical Gaussian noise, described as Zi ~ N{0,ar) and Zj ~ Ai'(0, of), 
and evaluated expectations E[{Zi)^] and We then obtained 

{ccU\ + q), (AI7) 



(1 - ccUU)^ 
ac 

(I - c£UU) 



E[iZ,f] = af = —^^-^(c£U'q + q), (AI8) 



where 



'?=^E(-^"V (A19) 



cJV 



l-^JK-rr (A20) 



cN 

From the self-averaging property, we obtained the SCSNA order parameter equation in sec 11111 

APPENDIX B: CORRELATION WITH N-STEP PREVIOUS STATE 

To evaluate the effect of the previous crosstalk noises, we must derive the correlation of a unit between the current 
state and the n-step before state. These are described by q2t+i,2(t-n)+i and q2t,2{t-n), respectively: 

92t + l,2(t-n) + l — 

J Dzexp(-2:^S-'2:){f2f+i(2:i)y2(i-n)+i(^2)) 
27r|E| 

<l2t,2(t-n) = 

/i?Zexp(-Z^S-^z)(y2t(gl)y2(t-n)(g2)) 



The matrices S, S and vectors z, z are described as follows: 

fit f2t,2(t-n) 
f2t,2(t-n) ?2{t-n) 

Zl 
Z2 



''2t-l r2t-l,2(t-n)-l 
1'2t,2(t-n) r2(t-n)-l 

Zl 
Z2 



(B2) 



The diagonal components of each matrix correspond to the variances of the current state noises. The non-diagonal 
components express the noise correlation between the current state and the ry-step previous state. For f] > 2, the 
correlations are described as: 

'''2t+l,2{t-r]) + l =Cg2t+l,2(t-)7+l) + l 

-I- CcU2t+lU2tr2t-l,2{t-r)+l) + l, 
f2t.2{t-r,) =Cq2t,2(t--q) 

+ CcU2tU2t-lf2{t-l),2{t-n)^ (B3) 



^2t+l,2(t-j)) + l — Cg2t+l,2{t-A)+l 
+ CC^U2t+lU2(t-X)+l<l2t,2(t-X) 



+ C^(cc)'^g2(t-A)+l,2(t-j,)+l Y\ U2T+1U2T 

n — r] t 

+ C^(cc)^g2t+l,2(t-i,-A)+l ]^ '?2(r-J7)+lt^2(T 
A=l T=t-A+1 

+ Cc'^ty2t+lU2(t-n)+l 

n—1 t 

X](cc)^g2(t-A),2(t-,,) Yi U2tU2t-1 
A=l T=t-A+1 
n — 1 — 77 

+ CC^[/'2t+l[/'2(t-,,)+l ^ (cc)'^g2t,2(t-j,-A) 
A=l 

t 

fj^ t^2(T-77)f^2(T-»7)-l 

T=t-A+1 

+ (c£)"(^2t+l'^2t'^2{t-77) + lf^2{t-)7)''2t-l,2{t-A)-l 



r2t,2{t—n) = cq2t,2(t-n) 

+ CC^U2tU2^t-r,)q2t-l,2(t-n)-l 
n t 

+ (^'^i(^)^Q2(t-X),2(t-v) II U^rU2T-l 
A=l T=t-A+1 

n — rj t 

+ (^'^{(^)^'l2t,2(t-V->^) II U2(T-n)U2(T-ri)-l 
A=l T=t-A+1 

+ CcU2tU2{t-n) 

n-1 t 

X^(cc)^g2(t-A)-l,2(t-,,)-l H [^2T-lt/2T- 
A=l T=t-A+1 

n— 1 — 77 

+ CcU2tU2(t-n) ^ (cc)'^g2t-l,2(t-T,-A)-l 



II U2(r-r,)-lU2(T-r,)-2 

T=t-A+1 

+ (cc)^C/2t(72t-lC/2(t-J7)f^2(t-J7)-l'^2(t-l),2(t-A-l) 
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1st layer 2nd layer 

FIG. 1: Network structure of BAM 



'SCSNA/theory' 
Simulation 



0.05 0.1 0.15 0.2 0.25 0.3 

a 



FIG. 2: Comparing SCSNA results with those of the simulation. The horizontal axis means the loading rate a, and 
the vertical axis means the overlap. The results of computer simulation axe shown as error-bars, which indicates 
median with minimum and maximum values. 
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^f^nd step approximation 
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(a) Simulation 



(b) 1-step analysis 



(c) 2-step analysis 
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(d) 3-step analysis 



(e) full-step analysis 



FIG. 3: Retrival process of a computer simulation and the statistical neurodynamics. The horizontal axis means time 
index t, and the vertical axis means the overlap m. (a) shows a result of computer simulation. Prom (b) to (e) shows 
the results of statistical neurodynamics. 
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FIG. 4: Capacity comparing the statistical neurodynamics with SCSNA. The horizontal axis means the loading rate 
a, and the vertical axis means the overlap m. The dashed curves shows the results of the statistical neurodynamics. 
The results of computer simulations are shown with error-bar which indicates mean with standard deviations. 



